Merging cache linefill

ABSTRACT

A data processing system  2  is described including a processor core  4  and a cache memory  6 . A controller  16  operates to allow data accesses to a cache line for which a pending cache linefill operation exists to be serviced for those data words within the cache line that are valid at the particular point in time. The old data may be manipulated up to the point where the first new data is returned. New data items may be read once they have been written into a victim cache line  18  even though the cache linefill is not completed. Stores to the victim cache line  18  may be made via a fill buffer  12  even though the linefill is still pending.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] This invention relates to the field of data processing systems.More particularly, this invention relates to data processing systemsincorporating a cache memory and for which data processing operationscontinue, where possible, when a cache miss occurs.

[0003] 2. Description of the Prior Art

[0004] It is known to provide data processing systems having cachememories in order to increase the processing performance of suchsystems. Cache memories provide a relatively small amount of high speedmemory compared with the main memory, such as RAM or disk storage, thatprovides a comparatively large amount of low speed storage. Whilstefforts are made to seek to provide within the cache memory a copy ofthe data desired from the main memory, inevitably data access requestsoccur to data items not currently stored within the cache memory. Insuch circumstances, a cache miss is triggered and typically a linefilloperation performed. A linefill operation selects a victim cache linecontaining multiple data items from within the cache memory and replacesthose data items with data items from a block of memory addressesincluding the desired data item that gave rise to the cache miss.

[0005] In known cache memory systems, such as the ARM920 processorproduced by ARM Limited, Cambridge, England, when a cache miss occurs,then the victim cache line is marked as invalid throughout thesubsequent linefill operation until the linefill is completed and thenew cache line is again marked as valid.

[0006] It is an object of the present invention to provide greaterprocessing performance within a data processing system including aprocessor core and a cache memory.

SUMMARY OF THE INVENTION

[0007] Viewed from one aspect, the present invention provides anapparatus for processing data, said apparatus comprising:

[0008] a processor core operable to generate a data access request to atarget data item having a target memory address;

[0009] a cache memory having a plurality of cache lines, each cache linestoring a plurality data items associated with an address value, saidcache memory being operable such that:

[0010] (i) if said target data item is stored within said cache memory,then said cache memory is operable to service said data access request;or

[0011] (ii) if said target data item is not stored within said cachememory, then said cache memory triggers a cache line fill operationwhereby a plurality of new data items including said target data itemare fetched from a main memory and written into a victim cache lineamong said plurality of cache lines replacing any old data itemspreviously stored in said victim cache line; and

[0012] a controller responsive to one or more status bits associatedwith said victim cache line to be operable during a cache line filloperation to permit data access requests to those data items storedwithin said victim cache line associated with a current address valuefor said victim cache line.

[0013] The invention recognises that during a linefill operation it ispossible to continue using the old data items for some time and also usenew data items from a partially completed linefill even though thelinefill operation is still in progress. Given the relatively long delayincurred by a linefill operation due to the relatively slow access thatis possible to the main memory, the processor core and cache memory maybe capable of performing a considerable amount of useful processingactivity during the linefill operation. The invention recognises thatthis processing activity can be extended to include accesses to thevictim cache line. As an example, until the first of the new data startsto be returned from the main memory, the old data in the cache line maycontinue to be made available to the processor core should subsequentdata processing operations require that data and there not be anyintervening dependency upon the operations that have missed thatprevents the subsequent data processing operation being performed. Thus,the old data in the victim cache line is preferably made available forcontinued use until its replacement within the cache actually starts.

[0014] In a similar manner, as the new data items are returned from themain memory, they are written into the victim cache line and it may bethat subsequent data processing operations can make use of those dataitems to carry out useful processing tasks even before the cachelinefill has completed.

[0015] This ability to continue to work with data items that are subjectto a pending cache linefill is further enhanced in embodiments thatprovide a fill buffer that may be associated with the victim cache andinto which new data values that are intended to overwrite the datavalues to be read from the main memory may be written even though thedata items from the cache miss have not yet been returned.

[0016] Whilst it would be possible to associate a plurality of statusbits with each cache line indicating the progress through a linefill forthat individual cache line, circuit area is saved when each cache lineis associated with a cache line status bit that serves to indicate thata linefill is pending for that cache line without in itself givinginformation about the progress of that particular linefill.

[0017] In circumstances in which only a single pending linefill isallowed at any given time, a single set of data item status bits may beprovided for the cache memory and serve to indicate the progress throughthe pending linefill operation with the cache line concerned beingidentified through another bit.

[0018] The bus requirements for storing out dirty cache data can bereduced in preferred embodiments in which the dirty data items areseparately removed from the victim cache line as they are replaced bytheir respective new data items.

[0019] In embodiments having both a set of data item status bits and acache line status bit, preferred embodiments act to first check whethera cache line status bit is set indicating a pending linefill operationand then reference is made to the data item status bits to determine thestate of progress of that linefill operation in order to grant or denypermission to a particular data access request being made to the cachememory.

[0020] It will be appreciated that the principle of the presentinvention could be applied to many different types of cachearchitecture. In particular, the invention is equally applicable tocontent addressable memory (CAM) architectures and TAG RAMarchitectures.

[0021] In the case of a content addressable memory architecture, it isconvenient to include the cache line status bits within the contentaddressable memory as then the check for a pending linefill operationcan be combined with the address matching operation normally performedand provided for as part of the content addressable memory operation.

[0022] The invention is particularly well suited for use in systems thatprovide hit-under-miss operation for their caches and also critical datafirst linefill operations.

[0023] Viewed from another aspect, the present invention provides amethod of processing data, said method comprising the steps of:

[0024] generating a data access request to a target data item having atarget memory address;

[0025] storing data items in a cache memory having a plurality of cachelines, each cache line storing a plurality data items associated with anaddress value, said cache memory being operable such that:

[0026] (i) if said target data item is stored within said cache memory,then said cache memory is operable to service said data access request;or

[0027] (ii) if said target data item is not stored within said cachememory, then said cache memory triggers a cache line fill operationwhereby a plurality of new data items including said target data itemare fetched from a main memory and written into a victim cache lineamong said plurality of cache lines replacing any old data itemspreviously stored in said victim cache line; and

[0028] in response to one or more status bits associated with saidvictim cache line, permitting during a cache line fill operation dataaccess requests to those data items stored within said victim cache lineassociated with a current address value for said victim cache line.

[0029] The above, and other objects, features and advantages of thisinvention will be apparent from the following detailed description ofillustrative embodiments which is to be read in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0030]FIG. 1 schematically illustrates a data processing systemincluding a cache memory and a processor core;

[0031]FIG. 2 schematically illustrates a victim cache line with itsassociated status bits and fill buffer;

[0032]FIG. 3 is an example sequence of operations illustrating alinefill operation in accordance with the present techniques; and

[0033]FIG. 4 is a circuit diagram illustrating a fill buffer and itsassociated connections with a bus interface unit, a cache and aprocessor core.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034]FIG. 1 schematically illustrates a data processing system 2including a processor core 4 and a cache memory 6. The cache memory 6 isconnected to a main memory 8, which could be a random access memory, ahard disk drive or some other form of bulk storage, via a bus interfaceunit 10 and a fill buffer 12. A write buffer 14 is also provided betweenthe cache memory 6 and the main memory 8. A cache controller 16 servesto manage the operation of the cache memory, such as providing victimselection for the cache lines upon a miss, control of linefill, controlof cache streaming and control of access to data within a cache linethat is subject to a pending linefill operation in accordance with thetechniques discussed above.

[0035] In operation, the processor core 4 executes data processinginstructions that may include loads from memory locations and stores tomemory locations. If the cache memory system is efficiently designed andutilised, then the vast majority of these data accesses may be servicedby data held within the cache memory 6. However, inevitably some loaddata access misses will occur when a data access is requested to a dataitem that is not residing within the cache memory 6. In suchcircumstances, the cache controller 16 serves to initiate a linefilloperation by selecting a victim cache line that is to be replaced andretrieving data to fill that victim cache line from the main memory 8,that retrieved data including the data item that gave rise to the miss.Victim selection, linefill control and cache streaming are in themselvesknown techniques within the field and will not be described furtherherein.

[0036] If the cache data to be replaced has been altered whilst it wasstored within the cache memory 6 compared to the copy of that data inthe main memory 8, then that data is termed “dirty” and needs to bewritten back to the main memory 8 in order to preserve integrity andensure proper operation. This “dirty” data may be stored out to a writebuffer 14 for writing back to the main memory 8 when time allows. It isgenerally more efficient to service the data access that gave rise tothe miss before completing any write back of “dirty” data and the writebuffer allows this capability.

[0037]FIG. 2 schematically illustrates a victim cache line 18 and itsinteraction with other parts of the system. In this example, the victimcache line 18 includes four data words DW1, DW2, DW3 and DW4 that aresequentially recovered from the main memory 8 during a linefilloperation. The particular data word that gave rise to the miss isrecovered first and streamed in parallel to the processor core 4 as itis returned and written into the cache 6 so that the processor core 4may resume operation as quickly as possible. Subsequent data words areretrieved in sequence with a wrap being performed as necessary.

[0038] The victim cache line 18 includes a content addressable memoryportion 20. For each cache line within the cache memory 6, the contentaddressable memory 20 stores address data that depends upon the size andassociativity of the cache memory 8 in the normal way. Together with theaddress data, there is also stored a valid bit 22 and a cache linestatus bit 24. The valid bit 22 serves to indicate whether or not thecache line concerned stores valid data, e.g. has been initialised aftersystem start-up. The use of a valid bit for a cache line is known in thefield. The cache line status bit 24 provides the function of indicatingwhether or not a pending linefill operation exists for the cache lineconcerned. This bit is set to a “1” at the time that the first new dataword is written into the victim cache line 18 and remains as a “1” untilthe final new data item has been written into the victim cache line 18,after which the value is returned to a 0 to indicate that the linefilloperation has completed.

[0039] The cache controller 16 includes a 4-bit register including dataitems status bits with one data item status bit corresponding to each ofthe data words. These status bits are set by the cache controller 16 andreferenced by the cache controller 16 to record and monitor the progressof a partially completed pending linefill operation.

[0040] In operation, when a data access request is received from theprocessor core 4, the memory address associated with that request iscompared with the value stored in the CAM portion 20 of the cache memory6 for each of the cache lines. If one of the cache lines gives a match,then the cache line status bit 24 for that cache line is examined to seeif a pending linefill operation is in place for that cache line. Thecache line status bit 24 will only be set once the first new data valuehas been written into the victim cache line 18. Accordingly, should thefirst new data value not yet have been returned, then the victim cacheline 18 will be treated as if it did not have a pending linefilloperation for it and the data access request concerned will be servicedin the normal way. If the cache line status bit 24 was set and that dataaccess request is a read (since the hit occurred the read must be to thenew data), then the cache controller 16 makes reference to the data itemstatus bits it stores itself to determine whether or not then particulardata item to which the data access read relates has yet been writteninto the victim cache line 18. If the new data item is present, then theread is allowed even though the rest of the linefill operation is stillpending. If the new data item is not in place, then the access requestwill stall until the data becomes available.

[0041] The fill buffer 12 provides the capability to service writerequests to the data words of the victim cache line 18 even whilst thelinefill for that cache line may be pending. A data access write requestwill be stored into the respective portion of the fill buffer 12 priorto being written into the corresponding data word of the victim cacheline 18 after that data word has been returned.

[0042]FIG. 3 illustrates an example of a linefill operation. At thefirst stage, the cache line illustrated contains valid data from addressA with no pending linefill. The data concerned is data values 2, 6, 1and 7. The fill buffer does not contain any required information at thisstage. At this initial time, the external memory at address location Bcontains data values 3, 5, 8 and 9.

[0043] At time (i), the processor core 4 issues a load request toaddress B that misses in the cache memory 6. This initiates a cachelinefill operation with the victim cache line being the one illustratedstoring the data values from memory address A.

[0044] At time (ii), the bus interface unit 10 returns the first dataword of the linefill (critical word first), namely data value 3. Thisdata value is stored into the fill buffer 12.

[0045] At time (iii), the address value for the victim cache line isupdated to B, the cache line status bit is set to “1” and the firstreturned new data value 3 is written into the first location within thevictim cache line 18. It will be appreciated that between times (i) and(iii), data access requests to the old data within the victim cache linemay be serviced normally since the CAM still contains the old addressdata value A and the cache line still contains the corresponding datawords. Also at time (iii), the second data word arrives back from themain memory 8 via the bus interface unit and is stored into the secondposition within the fill buffer. This second data word in this exampleis 5.

[0046] At time (iv), the second data word is transferred from the fillbuffer 12 to the victim cache line and the third data word is recoveredfrom the main memory and stored into the fill buffer 12. Between times(iii) and (iv), any data access read or write request made to the firstdata word 3 may be serviced since the correct memory address is presentwithin the CAM and the correct data value is present within the victimcache line 18. The ability to service this request is indicated by theset line status bit 24 and the first of the data item status bits withinthe cache controller 16 being set.

[0047] At time (v), the third data word 8 is transferred from the fillbuffer into the victim cache and the fourth data word returned from themain memory 8 via the bus interface unit 10. Between times (iv) and (v),both the first data word 3 and the second data word 5 may be subject toread or write requests that will be serviced even though the linefill ispending.

[0048] At time (vi), the final data word 9 is transferred from the fillbuffer into the victim cache line. Between times (v) and (vi), the firstthree data words namely 3, 5 and 8 may be subject to serviced read orwrite requests if these occur.

[0049] At time (iii), the first data word is being written into thevictim cache line 18 overwriting the old data word 2. At this time, thevictim cache line 18 is assumed to be dirty and the old data word iswritten out to the write buffer 14 under control of the bus interfaceunit 10. At the same time, a detection is made as to whether or not thevictim cache line 18 is truly dirty. If the victim cache line 18 was notdirty, then the write back request may be suppressed for the first datavalue. As each subsequent data value within the victim cache line 18 isreplaced by its new data value, the old data value, if dirty, is writtenout to be returned to the main memory 8.

[0050]FIG. 4 schematically illustrates the various interconnectionsbetween the fill buffer 12, the bus interface unit 10, the processorcore 4 and the cache memory 6. As will be seen, the fill buffer 12 isconnected such that it can receive stores to data words that may beselected by the illustrated multiplexers and latched within theappropriate latch portion of the fill buffer 12 whilst a linefillrequest is pending for a victim cache line 18. The latch 26 provides astorage location for a data value that is to be returned from the mainmemory 8 and is required to service an instruction previously issued bythe processor core 4 even though subsequent store instructions may haveoverwritten its position within the fill buffer 12.

[0051] The example illustrated shows a cache memory employing a contentaddressable memory architecture. It will be appreciated that thetechnique of the present invention would be equally applicable to a TAGRAM cache architecture and the modification of the technique describedherein to that architecture will be apparent to those skilled in theart.

[0052] It will also be appreciated that the embodiment providing asingle cache line status bit for each cache line with shared data itemstatus bits within the cache controller 16 is a useful way of reducingcircuit area requirements in systems where only a single cache linefillmay be pending at any given time. If multiple cache linefills may bepending at any given time, then individual data item status bits may beprovided for each cache line in order to track the progress of a cachelinefill for each cache line. If individual data item status bits areprovided for a cache line, then the cache line status bit may itself beomitted as it would then be redundant.

[0053] Although illustrative embodiments of the invention have beendescribed in detail herein with reference to the accompanying drawings,it is to be understood that the invention is not limited to thoseprecise embodiments, and that various changes and modifications can beeffected therein by one skilled in the art without departing from thescope and spirit of the invention as defined by the appended claims.

I claim:
 1. Apparatus for processing data, said apparatus comprising: aprocessor core operable to generate a data access request to a targetdata item having a target memory address; a cache memory having aplurality of cache lines, each cache line storing a plurality data itemsassociated with an address value, said cache memory being operable suchthat: (i) if said target data item is stored within said cache memory,then said cache memory is operable to service said data access request;or (ii) if said target data item is not stored within said cache memory,then said cache memory triggers a cache line fill operation whereby aplurality of new data items including said target data item are fetchedfrom a main memory and written into a victim cache line among saidplurality of cache lines replacing any old data items previously storedin said victim cache line; and a controller responsive to one or morestatus bits associated with said victim cache line to be operable duringa cache line fill operation to permit data access requests to those dataitems stored within said victim cache line associated with a currentaddress value for said victim cache line.
 2. Apparatus as claimed inclaim 1, wherein until a first new data item is written to said victimcache line, said controller permits data access requests to all of saidold data items in said victim cache line.
 3. Apparatus as claimed inclaim 2, wherein a new address value is associated with said victimcache line and becomes said current address value when said first newdata item is written to said victim cache line.
 4. Apparatus as claimedin claim 1, wherein after a first new data item is written to saidvictim cache line, said controller permits data access read requests tothose new data items that have been written to said victim cache line.5. Apparatus as claimed in claim 4, wherein a new address value isassociated with said victim cache line and becomes said current addressvalue when said first new data item is written to said victim cacheline.
 6. Apparatus as claimed in claim 1, comprising a fill bufferassociated with said victim cache line, said controller permittingwrites via said fill buffer to new data items within said victim cacheline throughout said line fill operation.
 7. Apparatus as claimed inclaim 1, wherein said one or more status bits include at least one cacheline status bit for each cache line indicating whether a line filloperation is pending for that cache line.
 8. Apparatus as claimed inclaim 1, wherein said one or more status bits include a set of data itemstatus bits for said cache memory indicating which new data itemsassociated with a line fill operation have been written to said victimcache line.
 9. Apparatus as claimed in claim 1, wherein old data itemsare separately removed from said victim cache line as they are replacedby respective new data items.
 10. Apparatus as claimed in claim 7,wherein said one or more status bits include a set of data item statusbits for said cache memory indicating which new data items associatedwith a line fill operation have been written to said victim cache lineand said controller is operable upon a data access read request to acache line to check if said cache line status bit indicates a pendingline fill for said cache line and, if so, then to check said set of dataitem status bits to determine if said data item being requested has yetwritten to said cache line.
 11. Apparatus as claimed in claim 1, whereinsaid cache memory includes a content addressable memory storing saidaddress values associated with said plurality of cache lines. 12.Apparatus as claimed in claim 7, wherein said cache memory includes acontent addressable memory storing address values associated with saidplurality of cache lines and said cache line status bit for each cacheline is also stored within said content addressable memory associatedwith address data for that cache line.
 13. Apparatus as claimed in claim1, wherein said cache memory and said controller support hit under missoperation.
 14. Apparatus as claimed in claim 1, wherein a first new dataitem is written to said victim cache line is said target data item, saidtarget data item being provided in parallel to said processor core. 15.Apparatus as claimed in claim 1, wherein said address value is a highorder portion of a memory address value.
 16. A method of processingdata, said method comprising the steps of: generating a data accessrequest to a target data item having a target memory address; storingdata items in a cache memory having a plurality of cache lines, eachcache line storing a plurality data items associated with an addressvalue, said cache memory being operable such that: (i) if said targetdata item is stored within said cache memory, then said cache memory isoperable to service said data access request; or (ii) if said targetdata item is not stored within said cache memory, then said cache memorytriggers a cache line fill operation whereby a plurality of new dataitems including said target data item are fetched from a main memory andwritten into a victim cache line among said plurality of cache linesreplacing any old data items previously stored in said victim cacheline; and in response to one or more status bits associated with saidvictim cache line, permitting during a cache line fill operation dataaccess requests to those data items stored within said victim cache lineassociated with a current address value for said victim cache line.